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Abstract. We formulate and establish a super duality which connects parabolic 
categories O for the ortho-symplectic Lie superalgebras and classical Lie algebras of 
BCD types. This provides a complete and conceptual solution of the irreducible 
character problem for the ortho-symplectic Lie superalgebras in a parabolic category 
O, which includes all finite dimensional irreducible modules, in terms of classical 
Kazhdan-Lusztig polynomials. 
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1. Introduction 

1.1. Finding the irreducible characters is a fundamental problem in representation 
theory. As a prototype of this problem, consider a complex semisimple Lie algebra g. 
The problem is solved in two steps, following the historical development: 

(1) The category of finite dimensional g-modules is semisimple and the correspond- 
ing irreducible g-characters are given by Weyl's character formula. 

(2) A general solution to the irreducible character problem in the BGG category O 
was given much later by the Kazhdan-Lusztig (KL) polynomials (theorems of 
Beilinson-Bernstein and Brylinski-Kashiwara) [KL, BB, BK]. 

1.2. The study of Lie superalgebras and their representations was largely motivated 
by the notion of supersymmetry in physics. A Killing-Cartan type classification of 
finite dimensional complex simple Lie superalgebras was achieved by Kac [Kl] in 1977. 
The most important subclass of the simple Lie superalgebras (called basic classical), 
including two infinite series of types A and osp, bears a close resemblance to the usual 
simple Lie algebras, so we can make sense of root systems, Dynkin diagrams, triangular 
decomposition, Cartan and Borel subalgebras, (parabolic) category O, and so on. 
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However, the representation theory of Lie superalgebras has encountered several 
substantial difficulties, as made clear by numerous works over the last three decades 
(cf. [BL, K2, KW, LSS, Pen, dHKT, dJ] as a sample of earlier literature, and some 
more recent references which can be found in the next paragraph): 

(1) There exist non-conjugate Borel subalgebras for a given Lie superalgebra g. 

(2) The category 3~ of finite dimensional g-modules is in general not semisimple. A 
uniform Weyl type finite dimensional character formula does not exist. 

(3) One has a notion of a Weyl group associated to the even subalgebra of g; 
however, the linkage in the category O (or in 3") of g-modules is not solely 
controlled by the Weyl group. 

(4) A block in the category O (or in 3~) may contain infinitely many simple objects. 

The conventional wisdom of solving the irreducible character problem for Lie superal- 
gebras has been to follow closely the two steps for Lie algebras in 1 . 1 . As the problem is 
already very difficult in the category 3", there has been little attempt in understanding 
the category O. 

For type A Lie superalgebra Qt(m\n), there have been several different general ap- 
proaches over the years. Serganova [Se] in 1996 developed a mixed geometric and 
algebraic approach to solving the irreducible character problem in the category 3\ 
Brundan in 2003 [Br] developed a new elegant purely algebraic solution to the same 
problem in 3~ using Lusztig-Kashiwara canonical basis. Developing the idea of super 
duality [CW] (which generalizes [CWZ]) which connects the categories O for Lie su- 
peralgebras and Lie algebras of type A for the first time, two of the authors [CL] very 
recently established the super duality conjecture therein. In particular they provided 
a complete solution to the irreducible character problem for a fairly general parabolic 
category O (including 3~ as a very special case) in terms of KL polynomials of type 
A. Independently, Brundan and Stroppel [BS] proved the super duality conjecture in 
[CWZ] , offering yet another solution of the irreducible character problem in 3~. 

1.3. The goal of this paper is to formulate and establish a super duality which connects 
parabolic category O for Lie superalgebra of type osp with parabolic category O for 
classical Lie algebras of types BCD, vastly generalizing the type A case of [CWZ, CW, 
CL]. In particular, it provides a complete solution of the irreducible character problem 
for osp in some suitable parabolic category O, which includes all finite dimensional 
irreducibles, in terms of parabolic KL polynomials of BCD types (cf. Deodhar [Deo]). 

1.4. Before launching on a detailed explanation of our main ideas below, it is help- 
ful to keep in mind the analogy that the ring of symmetric functions (or its super 
counterpart) in infinitely many variables carries more symmetries than in finitely many 
variables, and a truncation process can easily recover finitely many variables. The super 
duality can be morally thought as a categorification of the standard involution uj on 
the ring of symmetric functions and it only becomes manifest when the underlying Lie 
(super)algebras pass to infinite rank. Then truncation functors can be used to recover 
the finite rank cases which we are originally interested in. 

Even though the finite dimensional Lie superalgebras of type osp depend on two 
integers m and n, our view is to fix one and let the other, say n, vary, and so let us 
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denote an osp Lie superalgebra by Q n . By choosing appropriately a Borel and a Levi 
subalgebra of Q n , we formulate a suitable parabolic category n of g n -modules. It turns 
out that there are four natural choices one can make here which correspond to the four 
Dynkin diagrams b,b*,c, fl in 2.1 (the type a case has been treated in [CL]). There is 
a natural sequence of inclusions of Lie super algebras: 

01 C 02 C • • • C n C . . . . 

Let := \J^ = {Q n . A suitable category of 0-modules can be identified with the inverse 
limit limO n . On the other hand, we introduce truncation functors tc„ : Ofc — > n for 

k > n, as analogues of the truncation functors studied in algebraic group setting ( cf. 
Donkin [Don]). These truncation functors send parabolic Verma modules to parabolic 
Vermas or zero and irreducibles to irreducibles or zero. In particular, this allows us to 
derive the irreducible characters in n once we know those in 0. 

Corresponding to each of the above choices of g n and n , we have the Lie algebra 
counterparts g n and parabolic categories n for positive integers n. Moreover, we have 
natural inclusions of Lie algebras Q n C n +i> for all n, which allow us to define the Lie 
algebra := U n n and the parabolic category of 0-modules. Similarly, the category 
can be identified with the inverse limit limO n . In the main body of the paper, we 

actually replace 0,0 et cetera by their (trivial) central extensions. The reason is that 
the truncation functors depend implicitly on a stabilization scalar, which is interpreted 
conceptually as a level of representations with respect to the central extensions. 

To establish a connection between and 0, we introduce another infinite rank Lie 
superalgebra and its parabolic category 0. The Lie superalgebra contains and 
as natural subalgebras (though not as Levi subalgebras), and this enables us to 
introduce two natural functors T : — > and T : — s> 0. Using the technique of 
odd reflections among others, we establish in Section 4 a key property that T and 
T respect the parabolic Verma and irreducible modules, respectively. This result is 
already sufficient to provide a complete solution of irreducible osp-characters in the 
category n in the first half of the paper (by the end of Section 4.4). We remark that 
the idea of introducing an auxiliary Lie superalgebra and category has been used 
in the type A superalgebra setting [CL] . 

Recall that for the usual category O of Lie algebras, the KL polynomials were inter- 
preted by Vogan [Vo] in terms of Kostant u- homology groups. The u- homology groups 
make perfect sense for Lie superalgebras, and we may take this interpretation as the 
definition for the otherwise undefined KL polynomials in category O for Lie superalge- 
bras (cf. Section 1.2 (3)), as Serganova [Se] did in the category 3~ of 0l(m|n)-modules. 
In Section 4.5, we show that the functors T and T match the corresponding u-homology 
groups and hence the corresponding KL polynomials (compare [CKW]). Actually the 
computation in [CKW] of the u-homology groups with coefficients in the Lie superal- 
gebra oscillator modules via Howe duality was the first direct supporting evidence for 
the super duality for osp as formulated in this paper. 

Section 5 of the paper is devoted to proving that both T and T are indeed category 
equivalences. As a consequence, we have established that the categories and are 
equivalent, which is called super duality. A technical difference here from [CL] is that 
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we need to deal with the fact that parabolic Verma modules in may not have finite 
composition series. An immediate corollary of the super duality is that any known 
BGG resolution in the category gives rise to a BGG type resolution in the category 
0, and vice versa. 

1.5. The finite dimensional irreducible osp-modules are of highest weight, and they are 
classified in terms of the Dynkin labels by Kac [K2]. We note that the finite dimensional 
irreducible modules of non-integral highest weights are typical and so their characters 
are known [K2, Theorem 1]. It turns out that a more natural labeling of the remaining 
finite dimensional irreducible osp-modules (of integral highest weights) is given in terms 
of Young diagrams just as for classical Lie algebras (see e.g. [SW] for such a formulation 
and a new proof using odd reflections). 

As Borel subalgebras are not conjugate to each other, it becomes a nontrivial problem 
to find the extremal weights, i.e. highest weights with respect to different Borel, of a 
given finite dimensional irreducible osp-module. We provide an elegant and simple 
answer in terms of a combinatorial notion which we call block Frobenius coordinates 
associated to Young diagrams. 

We observe that our solution of the irreducible character problem in n includes 
solutions to all finite dimensional irreducible osp-characters. 

The category 3~ for a general osp (with the exception of osp(2|2n)) is not a high- 
est weight category and does not admit an abstract KL theory in the sense of Cline, 
Parshall and Scott [CPS], as indicated in the case of osp(3|2) [Ger, Section 2]. For 
a completely independent and different approach to the finite dimensional irreducible 
osp-characters in the category 3", see Gruson and Serganova [GS]. The finite dimen- 
sional irreducible characters of osp(2|2n) were obtained in [dJ]. The finite dimensional 
irreducible characters of osp(/c|2) were also computed in [SZ]. 

1.6. In hindsight, here is how our super duality approach overcomes the difficulties as 
listed in 1.2. 

(1) The existence of non-conjugate Borel subalgebras for a Lie super algebra is essen- 
tial for establishing the properties of the functors T and T. Choices of suitable 
Borel subalgebras are crucial for a formulation of the compatible sequence of 
categories n for n > 0. 

(2) The category 1 of finite dimensional g n -modules does not play any special 
role in our approach. Even the "natural" osp(M|2n)-modules C M ' 2n do not 
correspond well with each other under truncation functors, as they are natural 
with respect to the "wrong" Borel. 

(3) In the n — > oo limit, the linkage in the category of g-modules is completely 
controlled by the Weyl group of the corresponding Lie algebra q (which contains 
the even subalgebra of g as a subalgebra) . 

(4) In the n — > oo limit, it is no surprise for a block to contain infinitely many 
simple objects. 

In the extreme cases described in Section 2.3, we indeed obtain an equivalence of 
module categories between two classical (non-super!) Lie algebras of types C and D of 
infinite rank at opposite levels. If one is willing to regard osp(l|oo) as classical (recall 
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that the finite dimensional osp(l|2?i)-module category is semisimple) , there is another 
equivalence of categories which relates osp(l|oo) to the infinite rank Lie algebra of type 
B. In this sense, our super duality has a flavor of the Langlands duality. 

The super duality approach here can be further adapted to the setting of Kac- 
Moody superalgebras (including affine superalgebras) and this will shed new light on 
the irreducible character problem for these superalgebras. The details will appear 
elsewhere. 

It is well known that the proof of KL conjectures involves deep geometric machinery 
and results on D-modules of flag manifolds. The formulation of super duality suggests 
potential direct connections on the (super) geometric level behind the categories and 
0, which will be very important to develop. 

1.7. The paper is organized as follows. In Section 2 the Lie superalgebras g, g and g 
are defined, with their respective module categories 0, and introduced in Section 3. 
In Section 4, we provide a complete solution of the irreducible osp character problem 
in category n for all n, including all finite dimensional irreducible osp-characters, in 
terms of the KL polynomials of classical type. We establish in Section 5 equivalence 
of the categories and 0. Section 6 offers a diagrammatic description of the extremal 
weights of the finite dimensional irreducible osp-modules. 

Throughout the paper the symbols Z, N, and Z + stand for the sets of all, positive 
and non-negative integers, respectively. All vector spaces, algebras, tensor products, et 
cetera, are over the field of complex numbers C. 
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of Mathematics of University of Virginia for hospitality and support. The second 
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Mathematics of Academia Sinica in Taiwan for hospitality and support. The results of 
the paper were announced by the first author in the AMS meeting at Raleigh in April 
2009, and they were presented by the third author in conferences at Ottawa, Canada 
and Durham, UK in July 2009. 



2. Lie superalgebras of infinite rank 

In this section, we introduce infinite rank Lie (super)algebras g r , g f and g y associated 
to the 3 Dynkin diagrams in (2.2) below, where r denotes one of the four types b, b", c, 0. 

2.1. Dynkin diagrams of g r , g r and g ? . Let m € Z + . Consider the free abelian 
group with basis {e_ m , . . . ,e_i} U {e r \r € ^N}, with a symmetric bilinear form (•(•) 
given by 



(e r \e s ) = (-l) 2r 5 rs , r,s£ {—m, ... , —1} U -N. 
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We set 
(2.1) 



a x := e-i - e 1 / 2 , aj ■= tj - ej+i, -m < j < -2, 
/3 x :=e-i-ei, a r := e r - e r+1 / 2 , (3 r := e r - e r+ i, re^I 



For r = b, b" , c, 0, we denote by & the contragredient Lie (super) algebras ([Kl, Section 
2.5]) whose Dynkin diagrams (~W) together with certain distinguished sets of simple 
roots II(t r ) are listed as follows: 

b: 
b": 



c: 




o— o 



According to [Kl, Proposition 2.5.6] these Lie (super) algebras are so(2m+l), osp(l|2m), 
5p(2m) for m > 1 and so (2m) for m > 2, respectively. We will use the same notation 
(W) to denote the diagrams of all the degenerate cases for m = 0, 1 as well. (See 
Sections 2.2 and 2.3 below). We have used • to denote an odd non-isotropic simple 
root. So osp(l|2m) is actually a Lie superalgebra (instead of Lie algebra), but it is 
classical from the super duality viewpoint in this paper. 

For n € N let (JT^) and (jX^) denote the following Dynkin diagrams, where 

denotes an odd isotropic simple root: 



-o 

01 

(8)— ®- 

a X a l/2 



-o- 



-o- 

P3/2 




The Lie superalgebras associated with these Dynkin diagrams are g[(n + 1), g[(l|n) 
and g[(n|n + 1), respectively. In the limit n — > 00, the associated Lie superalgebras are 
direct limits of these Lie superalgebras, and we will simply drop 00 to write (~X) = 
C^oq) and so on. 

Anyof the head diagrams ( ¥ ) may be connected with the tail diagrams C%n), (X^) 
and (XtJ to produce the following Dynkin diagrams (n € N U {00}): 

(2.2) ® © ® © ® © 

_ — — — y 

We will denote the sets of simple roots of the above diagrams accordingly by Hn, LT n 
and LTn- For n = 00, we also denote the sets of positive roots by $ , and and 
the sets of roots by $ r , $ and <I> r , respectively. 
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2.2. Realization. Let us denote the 3 Dynkin diagrams of (2.2) at n = oo by (jF), 
(jF) and (jF). We provide a realization for the corresponding Lie superalgebras. 
For m G Z + consider the following totally ordered set I m 

3" _ T _ 1 3 

•<-<l<-<— 1< —2 < - - - < -m < < -m <■■■<— l<-<l<-<--- 
2 2 v . ' v v ' 2 2 

m m 

For ?n 6 Z + define the following subsets of I m : 



I m := { -1, . , -m , 0, r m, . . . , -1 } U {-, -, -, . . .} U {-, -, -, . . .}, 

m m 

~ 13 

Im : = {-m,...,-l, -,1, 2»2,...}- 

For a subset X of I m , define X x := X \ {0}, X+ := X n I+. 

2.2.1. General linear Lie superalgebra. For a homogeneous element v in a super vector 
space V = Vq © V\ we denote by its Z2-degree. 

For ?7i G consider the infinite dimensional super space V m over C with ordered 
basis {vi\i G I m }. We declare |v r | = |%| = 0, if r £ Z \ {0}, and \v r \ = \v ¥ \ = 1, 
if r € 2 + The parity of the vector ^ is to be specified. With respect to this 
basis a linear map on V m may be identified with a complex matrix (a rs ) ^ . The Lie 

superalgebra gl(y m ) is the Lie subalgebra of linear transformations on V m consisting 
of (a rs ) with a rs = for all but finitely many a rs 7 s. Denote by E rs G Ql(V m ) the 
elementary matrix with 1 at the rth row and sth column and zero elsewhere. 

The vector spaces V m and V m are defined to be subspaces of V m with ordered basis 
{v,i} indexed by l m and I m , respectively. The corresponding subspaces of V m , V m and 

V m with basis vectors V{, with i indexed by I m , I m and I m , respectively, are denoted 
by V^, F m and V^, respectively. This gives rise to Lie superalgebras gl(V m ), 0l(V m ), 
flW), gl(F^) andflTO). 

Let VF be one of the spaces Vm^V^ ,V m ,Vj^ ,V m or V m . The standard Cartan 
subalgebra of £|[(W) is spanned by the basis {E rr }, with corresponding dual basis {e r }, 
where r runs over the index sets I m , I m , I m , I m , I m , I m , respectively. 



2.2.2. Skew- super symmetric bilinear form on W . In this subsection we set |ug| = 1. 
For m G Z + define a non-degenerate skew-supersymmetric bilinear form (-|-) on V m by 

(2.3) (v r \v s ) = (vr\vs) = 0, (w r |i*) = 5 rs = -(-l)\ v ^\(vs\v r ), r,sel+, 

= !i (wffl"r) = (wol«r) =0, r G 1+ . 



Restricting the form to V^, V^, V^, V m and V^j^ gives rise to non-degenerate skew- 
supersymmetric bilinear forms that will again be denoted by (•]■). 
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Let W be as before. The Lie superalgebra spo(W) is the subalgebra of f(f(W) pre- 
serving the bilinear form (-|-). The standard Cartan subalgebra of spo(Vy) is spanned 
by the basis {E r := E rr — Er t r}, with corresponding dual basis {e r }- We have the 
following realizations of the corresponding Lie superalgebras for j: = b", c and m > 0: 



Table 1 



Lie superalgebra 


Dynkin diagram 


Lie superalgebra 


Dynkin diagram 


spo(V m ) 




spo(y m x ) 


(?) 


Spo(V m ) 






® 


5p0(V m ) 


8n 


spo(^) 


CD 



The sets IF, H and IF give rise to the following sets of positive roots: 

= {±e r -e s \r<s (r,s G 1+)} U {-2e % (i G 1+)} U {-e r (r Gl+)}, 
$V = {±er -e s \r < s (r, s G 1+ )} U {-2e, (i G 1+)}, 
$ b + = {±e t - €j\i < j (i,j G 1+)} U {-€i, -2e, (i G 1+)}, 
$V = {±£i " < i (»', J G 4)} U {-2 £i (t G 1+)}, 

¥+ = {±e r - e s |r < s (r,s G t)} U {-2e 4 (-m < i < -1)} U {-e r (r G 1+)}, 
¥+ = {±e r - e s \r < s (r, s G 1+)} U {-2e, (-m < i < -1)}. 

The corresponding subsets of simple roots can be read off from the corresponding 
diagrams in (2.2) (here we recall the notation of roots a's and /3's from (2.1)). 

2.2.3. Su^er symmetric bilinear form on W . Let W be as before. In this subsection 
we set \v-q\ = 0. Define a supersymmetric bilinear form (-|-) on V m by 

(2.4) (v r \v s ) = (vrlvg) = 0, (v r \vs) = 5 rs = (-l)M-KI r ,sGl+, 

(vq\vq) = 1, (uqI^) = (vq\v ¥ ) = 0, r G 1+ . 

Restricting the form to V^, V m , V£, V m and gives respective non-degenerate 
supersymmetric bilinear forms that will also be denoted by (-|-). The Lie superalgebra 
osp(W) is the subalgebra of 0t(VF) preserving the respective bilinear form determined 
by (2.4). The standard Cartan subalgebra of osp(W) is also spanned by the basis {E r := 
E rr — E 7t f}, with corresponding dual basis {e r }. We have the following realizations of 
the corresponding Lie superalgebras for j: = b,t> and m > 0: 



Table 2 



Lie superalgebra 


Dynkin diagram 


Lie superalgebra 


Dynkin diagram 


osp(V m ) 


m 






05p{V m ) 


® 


osp(V£) 




osp{V m ) 
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The sets IF, II r and IF give rise to the following sets of positive roots: 





= {±e r 


-e s \r<s (r,s£l+)}U{- 


-2e s 


( s6 I+) }U {-e r 




= {±e r 


-e s \r < s (r,s Gl+)}U{- 


-2e s 


(« G To + )}> 




= {±Q 


< J (iJ el+)}u{- 


e« (i 






= {±£i 


- cjl* < j (*, j e l+)}, 






< 


= {±£r 


-e,|r<s (r,set)}U{- 


-2e s 


(a € if)} U {-e r 




= {±£r 


-e,|r<s (r,set)}U{- 


-2e s 


(sett)}. 



Again, the subsets of simple roots can be read off from the corresponding diagrams in 
(2-2). 



2.3. The case m = 0. The Dynkin diagrams of spo(Ty) with a distinguished set of 

simple roots, for W = Vo, V X , Vq, Vq , Vq, Vq are listed in order as follows (see also 
Remark 2.1 below): 



O^® <g> ® (g^Hg) 

— 61/2 a l/2 a\ O r _l/2 «r a r+l/2 



"1/2 



£ l/2 - e l 




Hg) (g^Hg) 

ai "3/2 Or— 1/2 Or q t-+1/2 



-o— o 

/3„_i £!„ /3„+i 



— O-' 

61 /Si /3 2 



o^o — o 

-2ei /3 X |8 2 



-o— o 

P n -1 Pn Pn+1 



-o- 



-61/3 ft/2 03/2 ft-1 £r |8r4 
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For the sake of completeness we also list the corresponding sets of positive roots. 

={±e r -e s |r <s (r,s Gl+)}U{-2ei (iGl+)}U{-e r (r£l+)}, 
= {±e r - e s \r < s (r,s U {-2e 8 (i G 1+)}, 

= {±e 4 - ey|i < j (i, j G 1+)} U {- Ci , -2e, (i G 1+)}, 
= {±e, - e,-|t < j (ij G 1+)} U {-2ei (i G 1+)}, 

= {±e r -e s \r<s(r,s£ ij)} U {-e r (r G if)}, 

= {±e r -e s \r < s (r,s G Iq )}• 

Remark 2.1. It is easy to see that we have the following isomorphisms of Lie super- 
algebras with identical Dynkin diagrams: osp(Vo) — ^po(Vo), osp(Fo) — spo(Vo), 

0S p(y x ) - sP o(F x ), os p(F x ) =spo(y x ). 

2.4. Central extensions. We will replace the above matrix realization of the Lie 
superalgebras with Dynkin diagrams CW) and by their central extensions, 

for j; = b, b*, c,0. These central extensions will be convenient and conceptual for later 
formulation of truncation functors and super duality. 

Let m G Z+. Consider the central extension fl[(V m ) of gl(V m ) by the one-dimensional 
center CK determined by the 2-cocycle 

t(A,B) :=Str([S,A]B), A,Be 5 l(V m ), 

where 2 = + ^ — E rr and Str denotes the supertrace. Observe that the cocycle 

-2 ^ ~ ~ 

t is a coboundary. Indeed, as a vector space, 0t(l^n) = 0l(Vm) © CK, and let us denote 
by X for X G 0l(Vm,) to indicate that it is in Ql(V m ). Then the map from gl(V m ) to the 
direct sum of Lie superalgebras gl(V m ) © CK, which sends X to X' := X — Stv(3X)K, 
is an algebra isomorphism, i.e., pf', Y'] = [X, Y] +t(X, Y)K. 

For W = , V m , Vm , Vm, Vm the restrictions of r to the subalgebras 0l(W) give rise 
to respective central extensions, which in turn induce central extensions on osp(lF) and 
spo(VF). We denote such a central extension of spo(VF) or osp(W) by r (respectively, r 
and r ) when it corresponds to the Dynkin diagram (jF) in Tables 1 and 2 (respectively, 
(D and (?)). 

We make a trivial yet crucial observation that 0- r and r are naturally subalgebras of 
r . The standard Cartan subalgebras of f , y and y will be denoted by h r , f) and h/, 
respectively. f) r , b/ or h/ has a basis {.fT, -E r } with dual basis {Ao,e r } in the restricted 
dual (b/)*, (?)* or (b/)*, where r runs over the index sets 1L+, I m or I+, respectively. 
Here Ao is defined by letting 

A (K) = 1, A (E r ) = 0, 

for all relevant r in each case. 

In the remainder of the paper we shall drop the superscript j . For example, 
we write 0, and for r , r and r , with associated Dynkin diagrams (jp ) and 

(~0), respectively, where p denotes a fixed type among b, b", c, 0. 
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3. Categories 0, and 

In this section, we first introduce the categories 0, and of g-modules, g-modules 
and 0-modules, respectively. Then we study the truncation functors which relate g to 
finite dimensional Lie super algebras of osp type. 

Let m € Z + be fixed. 

3.1. The weights. We fix an arbitrary subset Yq of 11(6) . 

Let Y, Y and Y be the union of Yq and the subset of simple roots of (~T), (jZ) 
and respectively, with the leftmost one removed. We have Yq = for m = 0. 

As Y, Y and Y are fixed, we will make the convention of suppressing them from 
notations below. Set [, I and I to be the standard Levi subalgebras of q, g and g 
corresponding to the subsets Y, Y and Y, respectively. The Borel subalgebras of g, g 
and q, spanned by the central element K and upper triangular matrices, are denoted 
by b, b and b, respectively. Let p = [+ b,p = [+ b and p = I + b be the corresponding 
parabolic subalgebras with nilradicals u, u and u and opposite nilradicals u_, u_ and 
u_, respectively. 

Given a partition fx = (^1,^2, ■ • .), we denote by £(//) the length of \i and by // its 
conjugate partition. We also denote by 6(h) the modified Frobenius coordinates of \i: 

9(ji) := (e(Lj)i/2,O(rii,0(Hh/2,O(vh,...), 

where 

0((J,)i-i/2 := max{/ij — i + 1,0}, 0(fJ-)i '■= max{/Xj — i, 0}, i £ N. 

Let X- m , ■ ■ ■ , A-i G C and A + be a partition. The tuple (A_. m , . . . , A_i; A + ) is said to 
satisfy a dominant condition if (52i=- m ^ e i> e ^+ f° r an a e where h a denotes 
the coroot of a. Associated to such a dominant tuple and each d € C, we define the 
weights (which will be called dominant) 

-1 

(3.1) A := A ^ + E \ +e ; + dA o e ff, 

i=—m JGN 

-1 

(3.2) ^:=^A^+ (A + / s+ ie s +dA Gr, 

i=-m se|+Z+ 

-1 

(3.3) X 6 := Y X ^ + Yl °( X+ Ur + dA € %*. 

i=-m re i N 

We denote by P + C f)*, P + C b* and P + C b* the sets of all dominant weights of the 
form (3.1), (3.2) and (3.3) for all d £ C, respectively. By definition we have bijective 
maps 

^:P + ^P + , A i-> X\ 

9:P + ^P + , X i-> A e . 
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For fj, £ P + , let L(l, fi) denote the highest weight irreducible [-module of highest 
weight [i. We extend L(l, //) to a p-module by letting u act trivially. Define as usual 
the parabolic Verma module A(//) and its irreducible quotient L(n) over g: 

A( M ):=Ind{>£([,A0, A(/x) - L(/x). 

Similarly, for € P + , we define the irreducible I-module L(I, /Li 11 ), the parabolic Verma 
0-module A(/^) and its irreducible g-quotient L([r), as well as the irreducible [-module 
L(l,/ji 9 ), the parabolic Verma g-module A(// e ) and its irreducible g-quotient L(/i e ). 

3.2. The categories 0,0 and 6. 

Lemma 3.1. Let fi G P + . 

(i) JTie restrictions to [ of the Q-modules A(/i) and L(^) decompose into direct sums 
ofL(\,v) for v G P+. 

(ii) The restrictions to I of the g-modules A(/^) and L(^) decompose into direct 
sums of L(J, v^) for v G P + . 

(iii) The restrictions to I of the Q-modules A(/x e ) and L([i 8 ) decompose into direct 
sums of L([, u d ) for v G P + . 

Proof. Part (i) is clear. 

The proofs of (ii) and (iii) are analogous, and so we shall only give the proof for 
(ii). The I-module u_ is a direct sum of irreducible modules of the form L(J, u^). Now 
the category of [-modules that have an increasing composition series with composition 
factors isomorphic to L(l,ifi), with v G P + , is a semi-simple tensor category [CK, 
Section 3.2]. Thus A(/i^) = U(u~) ® L(J, fi^) also decomposes into a direct sum of 
L([, v*) with v G P + , and so does its irreducible quotient L{^). □ 

Let be the category of g-modules M such that M is a semisimple [)-module with 
finite dimensional weight subspaces M~, 7 G f)*, satisfying 

(i) M decomposes over [ into a direct sum of L(l,/i) for [i G P + . 

(ii) There exist finitely many weights Ai, A2, ■ ■ ■ , G P + (depending on M) such 
that if 7 is a weight in M, then 7 G Aj — ^ agn Z + a, for some i. 

The parabolic Verma modules A(//) and irreducible modules L(/i) for [i G lie in 
0, by Lemma 3.1. Analogously we define the categories and of g- and g-modules, 
respectively. They also contain suitable parabolic Verma and irreducible modules. The 
morphisms in 0, and are all (not necessarily even) g-, g- and g-homomorphisms, 
respectively. 

3.3. The Lie superalgebras g n , g n and g n of finite rank. For n G N, recall the 
sets n n ,n n ,n n of simple roots for the Dynkin diagrams (2.2). The associated Lie 
superalgebras g n , g n and g n can be identified naturally with the subalgebras of g, g 
and g generated by K and the root vectors of the corresponding Dynkin diagrams in 
(2.2), and moreover, g n C g n+ i,g n C g n+ i for all n. Observe that the g n 's (modulo 
the trivial central extensions) are exactly all the finite dimensional Lie superalgebras 
of osp type. Since g = U n g n , the standard Cartan subalgebra of g n equals fj n = fj ng n . 
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Similarly, we use the notation rj n and rj n for the standard Cartan subalgebras of g n and 
n , respectively. 

Recall the notation A G P+ \\ and A e from (3.1), (3.2) and (3.3). Given A G P + 
with A^ = for j > n, we may regard it as a weight A n G f)* in a natural way. 

Similarly, for A G P + with (A + )^- = for j > n, we regard A' 1 as a weight A n £ fj*. 
Finally, for A G P + with 6(X + )j = for j > n, we regard A e as a weight A n G ()* . The 
subsets of such weights A n , An, A„ in \f n , fj* and f)* will be denoted by P+, P+ and P+, 
respectively. 

The corresponding parabolic Verma and irreducible g n -modules are denoted by 
A n (/i) and L n (fi), respectively, with fi G P+, while the corresponding category of 
g n -modules is denoted by n . Similarly, we introduce the self-explanatory notations 
An(l^), LnQjfi), n , and A n (fj, e ), L n (fi e ), n for g n - and g n -modules, respectively. 

3.4. The truncation functors. Let oo > k > n. For M E Ok, we can write M = 
7 M 7 , where 7 runs over 7 G Yl7=-m ^- e « + So<j<fc Cej + CAo- The truncation functor 

tx k n :0 k ^0 n 

is defined by sending M to 0^^/^, summed over Yld^-m^i + So<j<n^-- e j + CAo- 
When it is clear from the context we shall also write ix n instead of ix n . Analogously, 
truncation functors ix n : — > n and ix n : Ok — > n are defined. 

Lemma 3.2. Let 00 > k > n and X = L, A. 

X n (m) , if (n, Ej ) = 0, Vj > re, 



(i) For fi G P^ 1 " we We tr n (X fc (/x)) 

(ii) For fi G P^ 1 " we Ziaue tx n (Xk(n)) 
(hi) Por G P^ we have tt n (Xk(fJ>)) 



0, otherwise. 

r X n (n), if{fi,Ej) = 0,Vj>n, 
0, otherwise. 

X n (aO , if (fJ,,Ej) = 0,Vj > n, 
0, otherwise. 



Proof. We will show (i) only. The proofs of (ii) and (iii) are similar. 

Since ix n otrj, = tt n , it is enough to show (i) for k = 00. Suppose that (//, P,) = for 
all j > n. Let [' denote the standard Levi subalgebra of g corresponding to the removal 
of the vertex f3 n of the Dynkin diagram of g. Then [' = g n © gl(oo). Now L(/i) is the 
unique irreducible quotient of the g-module obtained via parabolic induction from the 
['-module L n (/j,) (where the g n -module L n {fi) is extended to an ['-module by a trivial 
action of g[(oo)). Our choice of the Levi subalgebra and of the opposite nilradical 
assures that this parabolically induced module truncates to L n (fj,). Thus its irreducible 
quotient L(fi) also truncates to L n {n). The remaining case in (i) is clear. □ 

Remark 3.3. The central extensions introduced in Section 2.4 allow us to study, in 
a uniform fashion, modules whose weights stabilize at any d G C (not just at d = 
0). For example, if /U is a weight with /z(P r ) = d / 0, for r 3> 0, then, without 
central extensions, the usual truncation functors would always truncate an irreducible 
or parabolic Verma of such a highest weight to zero. A way around central extensions 
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is to define truncation functors depending on each d £ C. This approach, although 
equivalent, looks less elegant. 

4. The character formulas 

In this section we introduce two functors T : — > and T : — > 0, and then 
establish a fundamental property of these functors (Theorem 4.6). As a consequence, 
we obtain an irreducible osp-character formula in a parabolic category O in terms of the 
KL polynomials of BCD types (Theorem 4.8 and Remark 4.9). The Kazhdan-Lusztig 
polynomials in the categories 0, and in terms of Kostant u- homology groups are 
shown to match perfectly with one another (Theorem 4.13). 

4.1. Odd reflections. Let 9 be a Lie superalgebra with a Borel subalgebra 23 with 
corresponding sets of simple and positive roots n(23) and <J>+(23), respectively. As usual, 
for a positive root /3, we let fp denote a root vector associated to root —f3. 

Let a be an isotropic odd simple root in n(23) and h a be its corresponding coroot. 
The set <E> + (23 Q ) := {—a} U <3?+(23) \ {a} forms a new set of positive roots whose 
corresponding set of simple roots is 

n(s a ) = {/5 € n(S)|</3, K) = o, p ^ a} u {p + a\p e n(2$), (p, h a ) + 0} u {-«}. 

We shall denote by S Q the corresponding new Borel subalgebra. The process of such a 
change of Borel subalgebras is referred to as odd reflection with respect to a [LSS]. 

The following simple and fundamental lemma for odd reflections has been used by 
many authors (cf. e.g. [KW]). 

Lemma 4.1. Let L be a simple ^-module of "B -highest weight A and let v be a H -highest 
weight vector. Let a be a simple isotropic odd root in 14(25). 

(1) //(A, h a ) = 0, then L is a ^-module of H> a -highest weight A andv is a H a -highest 
weight vector. 

(2) // (A,/i a ) 0, then L is a Q-module of 55° -highest weight A — a and f a v is a 
¥> a -highest weight vector. 

4.2. The Borel subalgebras b c (n) and b s (n). Fix n S N. Starting with the third 

Dynkin diagram in (2.2) associated to q, we apply the following sequence of ^±12 odd 
reflections. First we apply one odd reflection corresponding to ai/2> then we apply two 
odd reflections corresponding to and «i/2 + «i + «3/2- After that we apply three 
odd reflections corresponding to 05/2; &3/2+ a 2 +^5/21 an d «i/2+ a i+ Q; 3/2+ a 2+tt5/2) e * 
cetera, until finally we apply n odd reflections corresponding to a n _!/2j «n-3/2 + a n-i + 
a n-i/2) ■ ■ ■ j Si=i 1 a i/2- The resulting new Borel subalgebra for q will be denoted by 
b c (n) and its corresponding simple roots are listed in the following Dynkin diagram: 

CD — GD— ®— o o — — ® 

01/2 Pn-1/2 a n + l/2 

The crucial point here is that the subdiagram to the left of the first (^) is the Dynkin 
diagram of Q n . 
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On the other hand, starting with the third Dynkin diagram in (2.2) associated to 
g, we apply the following new sequence of odd reflections. First we apply one 

odd reflection corresponding to a%, then we apply two odd reflections corresponding to 
a.2 and ai + a 3 / 2 + «2- After that we apply three odd reflections corresponding to 03, 
«2 + O5/2 + a 3i an( A ai + a 3 / 2 + «2 + "5/2 + a 3> e * cetera, until finally we apply n odd 
reflections corresponding to a n , a„,_i + a n -i/2 + a n, • • • , J2i=2 a i/2- The resulting new 
Borel subalgebra for g will be denoted by b s (n) and its corresponding simple roots are 
listed in the following Dynkin diagram: 

CD — (f^)— (§Co o — <g>^g> 

We remark that the subdiagram to the left of the odd simple root — Xa=2 a «/ 2 a b° ve 
becomes the Dynkin diagram of g n+1 . 

4.3. Highest weights with respect to b c (n) and b s (n). Recall the standard Levi 
subalgebra [ of g with (opposite) nilradical u and u_ (see Section 3.1). 

Lemma 4.2. The sequences of odd reflections in leave the sets of roots of u and 
U- invariant. 

Proof. This follows from the fact that the simple roots used in the sequences of odd 
reflections in Section 4.2 are all roots of I. □ 

We denote by b~(n) and t£(n) the Borel subalgebras of [ corresponding to the sets of 

simple roots n c (n) n ^2 a£ y ^ct and II s (n) n ^2 al ^_Za, respectively. The sequences of 
odd reflections in 4.2 only affect the tail diagram {/% ) and leaves the head diagram 
invariant. Since the tail diagram is of type A, the proofs of [CL, Lemma 3.2] and [CL, 
Corollary 3.3] can be adapted in a straightforward way to prove the following (where 
Lemma 4.2 is used). 

Proposition 4.3. Let A € P + and n G N. 

(i) Suppose that < n. Then the highest weight of L([, A ) with respect to the 
Borel subalgebra b~(n) is A. Furthermore, A(A e ) and L(\ s ) are highest weight 

Q-modules of highest weight A with respect to the new Borel subalgebra b c (n). 

(ii) Suppose that < n. Then the highest weight of L(l,X 9 ) with respect to the 
Borel subalgebra b~(n) is XK Furthermore, A(A e ) and L(\ 6 ) are highest weight 
Q-modules of highest weight with respect to the new Borel subalgebra b (n). 

4.4. The functors T and T. By definition, g and g are naturally subalgebras of g, 
[ and [ are subalgebras of I, while f) and b, are subalgebras of f). Also, we may regard 
f)* C f)* and Tj* C%*. 
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Given a semisimple [(-module M = ® 7g ^„ M 7 , we define 

T(M) := M 7 , and T(M) := M r 

Note that T{M) is an f)-submodule of the M, and T(M) is an f)-submodule of M. One 
checks that if M = © 7g ^» M 7 is an [-module, then T(M) is an [-submodule of M and 

T(M) is an [-submodule of M. Furthermore, if M = © 7g ^» M 7 is a g-module, then 

T(M) is a g-submodule of M and T{M) is a g-submodule of M. 

The direct sum decomposition in M gives rise to the natural projections 

7~ : M > T(M) and : M >■ T(M) 

that are f)- and {)-module homomorphisms, respectively. If / : M — > N is an f)- 
homomorphism, then the following induced maps 

T[j] : T(M) ► T(N) and T[f] : T(M) > T(N) 

are also f)- and f)-module homomorphisms, respectively. Also if / : M — > N is a g- 
homomorphism, then T~ and T[/] (respectively, Tj^ and T[/]) are g- (respectively, g-) 
homomorphisms . 

Lemma 4.4. For A G P+, we have T(L(J,\ 8 )) = L([,A), and T(L(T, A e )) = L(I, A 11 ). 

Proof. We shall prove the first formula using a character argument, and the second one 
can be proved similarly. 

Associated to partitions u C A, we denote by s^(xi,X2, ■ ■ •) and sa/i/( x i> x 2, ■ ■ •) the 
Schur and skew Schur functions in the variables xi,X2,.... The hook Schur functions 
associated to A is defined to be (cf. [Sv, BR]) 

(4.1) hs x (x 1/2 ,xi,x 3/ 2,x 2 , ...):= ^ s fl (x 1/2 , x 3/2 , ■ ■ .)s y/ ^(xi,x 2 , . . .). 

For a dominant tuple (A_ m , . . . , A_i; A + ), we have (cf. [CK]) 

(4.2) chL(T,A e ) = chL(lnt,X\t)hsy + (x l/2 ,xi,x 3/2 ,x 2 ,...). 

Here x r := e 6r for each r, and L(tnt, A|t) denotes the irreducible tnt-module of highest 

weight A|t = J27=-m x i e i- Note that Tnt = int. 

As an [-module, L([, A 61 ) is completely reducible. On the character level, applying T 
to L([, X e ) corresponds to setting Xi/ 2 ,x 3 / 2 ,x§/ 2 , ... in the character formula (4.2) to 

zero. By (4.1), T(L([, A 6 *)) is an [-module with character chL(l !n t, A|t) s\ + (x±, x 2 , ■ ■ •), 
which is precisely the character of L([, A). This proves the formula. □ 

Corollary 4.5. T and T define exact functors from to and from to 0, respec- 
tively. 

The following theorem can be regarded as a weak version of the super duality which 
is to be established in Theorem 5.4. 
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Theorem 4.6. Let A £ P + . If M is a highest weight Q-module of highest weight X , 
then T(M) and T(M) are highest weight g- and Q-modules of highest weights X and X\ 
respectively. Furthermore, we have 

T(A(X e ))=A(X), T(L(X e )) = L(A); 

T(A(X e )) = A(A^), T(L(X e )) =L(X*). 

Proof. We will prove only the statements involving T, and the statements involving T 
can be proved in the same way. 

By Proposition 4.3, M contains a b c (n)-highest weight vector v\ of highest weight 
A for n 3> 0. The vector v\ clearly lies in T(M), and it is a b-singular vector since 
b = g H b c (n). Now T(M) is completely reducible over [ with all highest weights of 
its irreducible summands lying in P + . Thus to show that T(M) is a highest weight 
g-module it remains to show that any vector in T(M) of weight in P + is contained in 
U(q)v\. This follows by the same argument in [CL, Lemma 3.5], which only relies on 
the A- type tail diagram of g. 

Let us write A(A) = E/(u_) <g>c L(l,X) and A(X d ) = E/(u_) ® c L(J, X e ). We observe 
that all the weights in U (u_), L(l, A), C7(u_), and L([, A 9 ) are of the form X]j<o a i e i + 
^ r>0 6 r e r with b r € Z + . Since also T(C/(u_)) = C/(u_), it follows by Lemma 4.4 that 
chT(A(A 6 ')) = chA(A). Since T(A(A 61 )) is a highest weight module of highest weight 
A, we have T(A{X 9 )) = A(A). 

To show that T sends irreducibles to irreducibles we show that T(L(X e )) has no 
singular vector apart from the scalar multiples of a highest weight vector. We argue 
by assuming otherwise and derive a contradiction. If we have another singular vector 
of weight different from A, then we can show, following the second part of the proof 
of [CL, Theorem 3.6], that we also have a singular vector in L(X e ) of weight different 
from X e . The argument there is applicable here, since it again only depends on the tail 
diagram, which is of type A. □ 

Remark 4.7. It can be shown that tilting modules exist in categories 0, 0, (cf. [CWZ, 
CW] for type A) and that the functors T and T respect the tilting modules. We choose 
not to develop the details in order to keep the paper to a reasonable length. 

By standard arguments Theorem 4.6 implies the following character formula. 

Theorem 4.8. Let X € P + , and write chL(A) = 2~^eP+ a A«AchA(/i), a^x G Z. Then 

(i) chI(A^) = E Me P+ a MAchA(^), 

(ii) chL(A e ) = E Me p + ^AchA(/). 

Remark 4.9. The transition matrix (a^x) in Theorem 4.8 is known according to the 
Kazhdan-Lusztig theory. This is because the Kazhdan-Lusztig polynomials in the BGG 
category O also determine the composition factors of generalized Verma modules in the 
corresponding parabolic subcategory (see e.g. [So, p. 445 and Proposition 7.5]). Hence 
Theorem 4.8 and Lemma 3.2 provide a complete solution to the irreducible character 
problem in the category n for the ortho-symplectic Lie superalgebras. 



18 



CHENG, LAM, AND WANG 



4.5. Kostant type homology formula. For a precise definition of homology groups 
of Lie superalgebras with coefficients in a module and a precise formula for the boundary 
operator we refer the reader to [CL, Section 4] or [Ta]. 

For M G we denote by Af = T(M) G and M = T(M) G 0. Furthermore let 
d : A(u_) <S> M — > A(u_) ® Af be the boundary operator of the complex of u_-homology 
groups with coefficients in Af , regarded as a u_-module. The map d is an [-module 
homomorphism and hence the homology groups iJ re (u_, Af ) are [-modules, for n G Z+. 
Accordingly we let d : A(u_) <g) Af ->• A(u_) ® M and d : A(u_) (g) Af -> A(u_) <g> M 
stand for the boundary operator of the complex of u_-homology with coefficients in 
Af and the boundary operator of the complex of u_-homology with coefficients in Af , 
respectively. Similarly, d and d are [- and [-homomorphisms, respectively. 

Lemma 4.10. For M G and A G P + , we have 

(i) T(A(u_) ® M) = A(u_) (8) M, and i/iws T(A(u_) ® L(A 9 )) = A(u_) ® L(A). 
Moreover, T[d] = d. 

(ii) T(A(u_) ® M) = A(u_) <8> M, and i/ms T(A(u_) ® Z(A e )) = A(u_) ® I(A h ). 
Moreover, T[d] = d. 

Proof. We will prove (i) only. It follows by definition of T and u„ that T(A(u_)) = 
A(u_). Now, since all modules involved have weights of the form X^<o a * e * + Sr>o ^ r€r 
with b r G Z+, it follows that T(A(u_) ® M) and A(u_) (8> Af have the same character. 
Complete reducibility of the [-modules T(A(u_) ® M) and A(u_) <8> Af implies that 

T(A(u_) ® Af) = A(u_) (g> M as [-modules. Theorem 4.6 completes the proof of the 
first part of (i). 

The second part of (i) follows from the definitions of d and d (see e.g. [CL, (4.1)]). □ 
By Lemma 4.10 we have the following commutative diagram. 



A n+1 (u_)<g>M — A n (u_)<g)M — ► A n_1 (u_) <8> Af ■ 



(4.3) 



T — 
A n (u_)®M 



T 1 

A n -!(u_)®M 



->• A n+1 (u_) <g> Af — A n (u_)(g)Af — I—* A n_1 (u_) <£> M ■ 



Thus T induces an [-homomorphism from if n (u_;Af) to if n (ii-;Af). Similarly, T 
induces an I- homomorphism from if n (u_; Af) to if n (u_; Af). 

As an [-module, A(u_) is a direct sum of L(l,fi"), fi G P + , each appearing with 
finite multiplicity ([CK, Section 3.2.3]). By [CK, Theorem 3.2], A(u_) <8> M as an T- 
module is completely reducible. Write A(u_) (8) Af = Mg p+ L(J,fi e ) m ^ as [-modules. 
It follows by Lemmas 4.4 and 4.10 that A(u_)<g>Af ^ MeP+ L([,/i) m M, as [-modules. 
Similarly, A(u_) ® M = AtgP + / u ll ) m ^, as I-modules. The commutativity of (4.3) 
and Lemma 4.10 now allow us to adapt the proof of [CL, Theorem 4.4] to prove the 
following. 

Theorem 4.11. We have for n > 
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(i) T(F n (u_;M)) ^ H n (u~;M), as {-modules. 

(ii) T(H n (u-;M)) ^ H n (u_;M), as [-modules. 

Setting M = L(X e ) in Theorem 4.11 and using Theorem 4.6 we obtain the following. 

Corollary 4.12. For A € P + and n > 0, we have 

(i) T(H n (u~;L(\ e ))) = H n (u-; L(\)) , as {-modules. 

(ii) T(H n (u-;L(\ e ))) ^ H n (u-;L(X^)), as [-modules. 

We define parabolic Kazhdan-Lusztig polynomials in the categories 0, and for 
/i,AG P + by letting 

oo 

W?) : = E dim c ( Hom i [ L 0, (u_; L(A))]^ (-g)"", 

n=0 
oo 

: = ^dim c (Hom I [L(I,^), J ff n (u_;L(^))])(-g)-", 

n=0 

oo 

W(<?) := E dim c( Hom r[ L ( T '^)'^("-^( Ae ))])(-'?)" n - 

n=0 

By Vogan's homological interpretation of the Kazhdan-Lusztig polynomials [Vo, Con- 
jecture 3.4] and the Kazhdan-Lusztig conjectures [KL], proved in [BB, BK], £fi\(q) co- 
incides with the original definition and moreover lu\(l) = (cf. Theorem 4.8). The 
following reformulation of Corollary 4.12 is a generalization of Theorem 4.8. 

Theorem 4.13. For A,/i6 P + we have £^\{q) = £ne\e{q) = i^x* (<?)• 

5. Equivalences of categories 

5.1. The following is standard (see, for example, [Ku, Lemma 2.1.10]). 

Proposition 5.1. Let M € 0. Then there exists a (possibly infinite) increasing filtra- 
tion = Mq C Mi C M2 C • • • of Q-modules such that 

(i) [)i> Q M i = M, 

(ii) Mil Mi— 1 is a highest weight module of highest weight z/j with Vi E P + , for i > 1, 

(iii) the condition Ui — Uj € X]«gn implies that i < j, 

(iv) for any weight fx of M , there exists an r € N such that (M/M r )^ = 0. 

Similar statements hold for M S and M € . 

Let denote the full subcategory of consisting of finitely generated U (g)-modules. 

The categories and J are defined in a similar fashion. 
Proposition 5.1 implies the following. 

Proposition 5.2. Let M € 0. Then M 6 0* if and only if there exists a finite 
increasing filtration = Mq C Mi C Mi C • • • C M^ = M of Q-modules such that 
Mi/Mi-i is a highest weight module of highest weight ui with € P + , for 1 < i < k. 
Similar statements hold for MeO and M € . 
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The following proposition is the converse to Theorem 4.6. 

Proposition 5.3. (i) If V(X) is a highest weight Q-module of highest weight X G 
P + , then there is a highest weight Q-module V(X e ) of highest weight X such 
that_T(V{X e )) = V{X). 
(ii) If V{X^) is a highest weight Q-module of highest weight X^ with X G P + , then 
there is a Q-module V(X e ) of highest weight X e such that T(V(X 6 )) = V(X^). 

Proof. We shall only prove (i), as (ii) is similar. We let W be the kernel of the natural 
projection from the A(A) to V(X). Now Theorem 4.6 says that T(A(A e )) = A(A). 
Thus, by the exactness of functor T, it suffices to prove that W lifts to a submodule 
W of A(X 9 ) such that T(W) = W. 

There is an increasing filtration = Wq C W\ C Wi C • • • of g-modules for W 
satisfying the properties of Proposition 5.1. For each i > 0, let Vi be a weight vector in 
Wi such that vi + Wj_i is a non-zero highest weight vector of Wi/Wi-\. Observe that 
A(A e ) = ®^ P +L(\^ 6 ) m ^ and A(A) = AteP+ L(l, n) m ^) are completely reducible 

[- and [-modules, respectively. Then, for each i > 0, there is a highest weight vector 
Vi of the [-module U(l)vi with respect to the Borel subalgebra b n I. Let W{ be the 
submodule of A(A 9 ) generated by vi,V2, • • • ,Vi and set Wq = 0. It is easy to see Vi 
is a highest weight vector of the g-module Wi/Wi-±. Let W = Ui>i^- ^ * s cl ear 
tha^T(Wi/Wi-i) =i Wt/Wi-i for all i. This implies T(Wi) = Wi for all i and hence 
T(W) = W. □ 

5.2. The categories O'' and ' . Define an equivalence relation ~ on f)* by letting 
fj, ~ v if and only if fj, — v lies in the root lattice Z$ of q. For each such equivalence class 
[fj], fix a representative [fj]° G [)* and declare [fj]° to have Z2-grading 0. For e = 0, 1, 
set (cf. [Br, §4-e] and [CL, Section 2.5] for type A) 

K = {^e?| E (^-M°)(^r) = e (mod2)}, forj = b,c,V, 

rel/2+Z+ 
m 

K = e ? | J> " + I> " M°)$0 = e (mod 2)}, for t = bV 

Recall that V G is a semisimple fj-module with V = © g p Ky. Then V acquires a 
natural Z2-grading V = Vq ® Vj given by 

(5.1) K==©^. e = M, 

which is compatible with the Z2-grading on g. 

We define 0° and O^ to be the full subcate gories of and 0* , respectively, consist- 
ing of objects with ^-gradation given by (5.1). Note that the morphisms in 0° and 

0^'° are of degree 0. For M G 0, let M G 0° denote the g-module M equipped with 

the Z2-gradation given by (5.1). It is clear that M is isomorphic to M in 0. Thus 
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and have isomorphic skeletons and hence they are equivalent categories. Similarly, 
f and f '° are equivalent categories. 

Analogously define 0^, 0^'°, 0° and 0^'° to be the respective full subcategories of 0, 
0^, and 0^ consisting of objects with Z2-gradation given by (5.1). Similarly, 0^ = 0, 

0° ^ 0, and also 0-^° ^ 0', L ° = / . (In case of and Qf , these remarks are trivial 
except for the type b° which corresponds to a Lie superalgebra). 

5.3. Equivalence of the categories. Recall the functors T and T from Section 4.4. 
The following is the main result of this section. 

Theorem 5.4. (i) T : — > is an equivalence of categories. 
(ii) T : — > is an equivalence of categories. 
Hence, the categories and are equivalent. 

Since 0° = and 0° = it is enough to prove Theorem 5.4 for 0° and 0°. In 
order to keep notation simple we will from now on drop the superscript and use 0,0, 
0^ and 0^ to denote the respective categories 0°, 0°, f '° and for the remainder 
of Section 5. Henceforth, when we write A(\ s ), L(X 9 ) € 0^, A G P + , we will mean 
the corresponding modules equipped with the ^-gradation (5.1). Similar convention 
applies to A(A^) and L(X^). 

For M,N G and % G N the ith extension group Extg(M, N) can be under- 
stood in the sense of Baer-Yoneda (see e.g. [Mit, Chapter VII]) and Extg(M, N) := 
Homo(Af, N). In a similar way extensions in and are interpreted. From this view- 
point the exact functors T and T induce natural maps on extensions by taking the 
projection of the corresponding exact sequences. 

Theorem 5.5. We have the following. 

(i) T : 0* — > 0? is an equivalence of categories. 

(ii) T : / -> ; is an equivalence of categories. 

, r 

(iii) The categories O-' and are equivalent. 

Theorem 5.5 can be proved following a similar strategy as the one used to prove 
[CL, Theorem 5.1]. To avoid repeating similar arguments at great length we will just 
point out the main differences between their proofs. In [CL, Section 5] the main point 
is to prove that the functor T induces isomorphisms Homg(M,A^) = Homo(M, iV) 
and Ext~(L, N) = Extg(L, N), for L irreducible, and M,N having finite composition 
series. From this [CL, Theorem 5.1] is derived easily. As the isomorphism of the Horn 
spaces imply the isomorphism of the Ext 1 spaces [CL, Lemma 5.12], we are reduced to 
establish the isomorphism of the Horn spaces. To prove the isomorphism of the Horn 
spaces therein, the idea is to prove this isomorphism first for M irreducible, and then 
to use induction on the length of the composition series of M to establish the general 
case. 

Now, thanks to Proposition 5.3, we can proceed similarly as in [CL, Section 5] to 
prove Theorem 5.5. For this purpose we replace L by a highest weight module, and 
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M and N by finitely generated modules. As finitely generated modules possess finite 
nitrations whose subquotients are highest weight modules (cf. Proposition 5.2), we 
can now borrow the same type of induction arguments from [CL], now inducting on 
the length of such a filtration instead of the length of a composition series. Therefore, 
the proof of the isomorphisms is again reduced to a special case, namely when M is a 
highest weight module. This case can then be proved using similar arguments as the 
ones given in the proof of [CL, Lemmas 5.8]. The case of T is completely analogous. 
Having Theorem 5.5 at our disposal we can now prove Theorem 5.4. 

Proof of Theorem 5.4- Since the proofs of (i) and (ii) are similar, we shall only prove 
(i). (iii) follows from (i) and (ii). For every M £ 0, there is an increasing filtration 
= Mo C Mi C M2 C • • • of 0-modules for M with Mj £ 0^ satisfying the properties 
of Proposition 5.1. The filtration {MJ of M lifts to a filtration {M} with M £ f 
such that T{Mi) = Mi by Theorem 5.5. It is clear that we have M := \J i>0 Mi € 
and T(M) M. 

It is well known that a full and faithful functor F : C 1— > C, satisfying the property 
that for every M' £ & there exists M £ C with F(M) = M', is an equivalence of 
categories (see e.g. [Pop, Proposition 1.5.2]). 

Therefore it remains to show that T is full and faithful. By Proposition 5.1, for 
M £ 0, we may choose an increasing filtration of g-modules = Mq C Mi C M2 C • • • 
such that U i>0 Mj = M and Mj/Mj_i is a highest weight module of highest weight vf 
with vi £ P + , for i > 1. Then the direct limit of M, is limMj = M and Homg(M, N) = 
hmHomg(Mj, N) for every N £ 6. Similarly we have limM, = M and Hom (M, N) = 

^im Homo (Mj , N) for N = T(N). Furthermore, we have the following commutative 
diagram (where ip = hm T~ ^): 



Using a similar argument as the one given in [CL, Lemma 5.10], where we replace 
the induction on the length of composition series therein by induction on the length 
of finite increasing filtration = Mq C Mi C M2 C • • • C Mj, we show that 
Ttt, fr : Homg(Mj, iV) —¥ Homo (Mj,iV) are isomorphisms for each i. Therefore (p 




M.N 




is an isomorphism and hence T- 



M,N 



is an isomorphism. This completes the proof. □ 



6. Finite dimensional representations 



The main purpose of this section is to determine the extremal weights of finite di- 
mensional irreducible modules over the ortho-symplectic Lie superalgebras with integral 
highest weights. It follows that all such finite dimensional irreducible modules for the 
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orthosymplectic Lie superalgebras are in the category n . We note that the finite di- 
mensional irreducible modules of non-integral highest weights are typical and so their 
characters are known [K2, Theorem 1]. 

6.1. Extremal weights for osp(2m + lj2n). Let us denote the weights of the natural 
osp(2m + l|2n)-module C 2n l 2m+1 by ±<5i,0, ±ej for 1 < i < n, 1 < j < m. We call a 
weight integral, if it lies in Z-span of the S^s and e^'s. The standard Borel subalgebra 
23 st of osp(2m + l\2n) is the one associated to the following set of simple roots 

o — O O^W^O 0^0 

Si — 82 $2 — &3 "5 n _i — 5 n £1 — £2 e m _i — e m e m 

An arbitrary Dynkin diagram for osp(2m + l\2n) always has a type A end while the 
other end is a short (even or odd) root. Starting from the type A end, the simple roots 
for a Borel subalgebra 23 of osp(2m + l|2n) give rise to a sequence of d\ 5's, e\ e's, di 
<5's, e2 e's, . . . , d r <5's, e r e's and sequences of ±l's: (£i)i<i< n U (Vj)i<j<m (all the d{ and 
ej are positive except possibly d\ = or e r = 0). Note that a Dynkin diagram contains 
a short odd root exactly when e r = 0. Let 

u u 

dy, — ^ d a , & u — ^ ^ e a 

a=l a=l 

for u = 1, . . . , r, and let do = eo = 0. Note d r = n, e r = m. More precisely, there exist 
a permutation s of {1, . . . , n} and a permutation t of {1, ... , m}, so that the simple 
roots for 23 are given by 

& s (i) ~ &+i<5 s (i+i), ig{d u \u = l,...,r}; 

Vj£t(j) ~ Vj+i£t(j+i), 1 < j < ™, j {e u \u = 1, . . . , r}; 

U u ^s(d u ) ~ Vi+B u . 1 e t{1+eu _ 1 ), for 1 < u < r if e r > (or 1 < u < r if e r = 0); 

VeJt( eu ) ~ ti+djs(i+d u ), u = 1, . . . ,r - 1; 

??e r e t ( er ), if e r > (or UJs(d r ) if e r = 0). 

Recall a partition A = (Ai, A2, . . .) is called an (n\m)-hook partition, if A n +i < m (cf. 
[BR, Sv]). For such a A, we define 

X* = (Ai,...,A n ,i/i,...,z/ TO ), 
where (yi, . . . , f m ) is the conjugated partition of (A n+ i, A n+ 2, . . .). 

Lemma 6.1. [K2] The irreducible osp(2m + l|2n) -module of integral highest weight 
Y27=l ^i^i+SjLi with respect to the standard Borel subalgebra is finite dimensional 
if and only if (Ai, . . . , A n , Ai, . . . , A m ) = A* for some (n\m)-hook partition A. 

We denote by L'(osp(2?n + l|2n), A*) these irreducible osp(2m + l|2n)-modules with 
respect to the standard Borel subalgebra, to distinguish from earlier notation used 
for irreducible modules with respect to different Borel subalgebra. Actually the finite 
dimensionality criterion was given in [K2] in terms of Dynkin labels, which is known 
to be equivalent to the more natural labeling above in terms of (ra|m)-hook partitions 
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(cf. [SW]). Same remark applies to the finite dimensionality criterion for osp(2m|2n) 
in Lemma 6.8 below. 

Example 6.2. Suppose that the corresponding Dynkin diagram of a Borel subalgebra 
of osp(9|10) is as follows: 



We read off from the above a signed sequence with indices 52(— #i)(— ei)e2<53<54e4e3(— 65). 
In particular, we obtain a sequence 55ee55ee5 by ignoring the signs and indices. In this 
case, d\ = d,2 = 2,^3 = 1, and e± = e<i = 2. Furthermore, the sequences (£j)i<i<5 and 
(ryj)i<j<4 are (1, —1, 1, 1, —1) and (—1, 1, 1, 1), respectively. 

Define the block Frobenius coordinates (pi\qj) of an (n|m)-hook partition A associated 
to 23 as follows. For 1 < i < n, 1 < j < m, let 

Pi = max{Aj — e u , 0}, if d u < i < d u+ \ for some < u < r — 1 

qj = max{Aj — d u+ i, 0}, if e u + 1 < j < e u+ \ for some < u < r — 1. 

It is elementary to read off the block Frobenius coordinates of A from the Young diagram 
of A in general, as illustrated by the next example. 

Example 6.3. Consider the (5,4)-hook diagram A = (14,11,8,8,7,4,3,2). The block 
Frobenius coordinates associated with 23 from Example 6.2 for A is: 

pi = 14,p 2 = 11, P3 =Pi = 6,J> 5 = 3; qi = q 2 = 6, q 3 = 3, q A = 2. 
These are read off from the Young diagram of A by following the e5 sequence 55ee55ee5 




O — ^K) — &^O^g^>Hg^« 




as follows: 



m 



= 4 



P2 — > 



^P3^ 



PA -> 



t t 



t t 



^P5 ~> 



11 



= 5- 



qi q2 



qz 14 

I r 1 



Theorem 6.4. Let A be an (n\m)-hook partition. Let 23 be a Borel subalgebra of 
osp(2m + l|2n) and retain the above notation. Then, the r B-highest weight of the simple 



SUPER DUALITY AND ORTHO-SYMPLECTIC LIE SUPERALGEBRAS 



25 



osp(2m + l\2n) -module L'(osp(2m + l|2n), A # ) 



IS 



i=l j=l 

Proof. Let us consider an odd reflection that changes a Borel subalgebra 23i to 232- 
Assume the theorem holds for 23i. We observe by Lemma 4.1 that the statement of the 
theorem for 23 2 follows from the validity of the theorem for 23i. The statement of the 
theorem is apparently consistent with a change of Borel subalgebras induced from a real 
reflection, and all Borel subalgebras are linked by a sequence of real and odd reflections. 
Hence, once we know the theorem holds for one particular Borel subalgebra, it holds 
for all. We finally note that the theorem holds for the standard Borel subalgebra 23 st , 
which corresponds to the sequence of n 5's followed by m e's with all signs £j and rjj 
being positive, i.e., A ss = A*. □ 

Example 6.5. With respect to the Borel 23 of osp(9|10) as in Example 6.2, the 23- 
extremal weig ht of L'(osp(9|10),A # ) for A as in Example 6.3 equals to 

-116! + U5 2 + 6<5 3 + 6<5 4 - 3<5 5 - 6ei + 6e 2 + 2e 3 + 3e 4 . 

Corollary 6.6. Every finite dimensional irreducible osp (2m + l\2n) -module of integral 
highest weight is self-contragradient. 

Proof. Denote by 23 op the opposite Borel to the standard one 23 st . It follows by Theo- 
rem 6.4 that the 23 op -extremal weight of the module L'(osp(2m+ l|2n), A*) is —A*. □ 

Recall that the following Dynkin diagram of 0Sp(2m-|-lj2n) and of its (trivial) central 
extension has been in use from the point of view of super duality and it is opposite 
to the one associated to the standard Borel 23 st . 



-0—0 o 

Q_ m a_ m+ i a_i /3 1/2 /3 3/2 /3„- 3 /2 

Setting €j = e_ m+ j_i and <5j = e n _i+i/2 to match the notation in this section with 
the one used earlier, we have the following immediate corollary of Lemma 6.1 and 
Theorem 6.4. 



Corollary 6.7. An irreducible integral highest weight osp(2m + l\2n)-module with re- 
spect to the Borel subalgebra con 
the highest weight is of the form 



sped to the Borel subalgebra corresponding to f 0« J * s finite dimensional if and only if 



(6.1) - max{A^- - n, 0} e_j - ^ A n _i +1 e i _ 1 / 2 , 

j=i i=i 

where A = (Ai, A2, • • •) is an {n\m)-hook partition. 
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6.2. Extremal weights for osp(2m|2n). Let us denote the weights of the natural 
osp(2m|2n)-module C 2n l 2m by ±5i,±€j for 1 < i < n, 1 < j < m. The standard Borel 
subalgebra 23 st of osp(2m|2ra) is the one associated to the following set of simple roots 

_ o— o o^g^-o o( 

8i — 82 52 — 83 5 n _ i — S n ei — 62 e m -2 — e m _i\^ 

Q 6n-l + Cm 

There are two kinds of Dynkin diagrams and corresponding Borel subalgebras for 
osp(2m|2n): 

(i) Diagrams of |-shape, i.e., Dynkin diagrams with a long simple root ±2<5j. 

(ii) Diagrams of Y-shape, i.e., Dynkin diagrams with no long simple root. 

We will follow the notation for osp(2m + l\2n) in Subsection 6.1 for sets of simple 
roots in terms of signed e5 sequences, so we have permutations s, t, and signs rjj. We 
fix an ambiguity on the choice of the sign rj m associated to a Borel B of Y-shape, by 
demanding the total number of negative signs among rjj(l < j < m) to be always even. 

Let A be an (njm)-hook partition, and let the block Frobenius coordinates (pi\qj) be 
as defined in Subsection 6.1. Introduce the following weights: 

n m 

t=l j=l 
n m— 1 

i=l j=l 

The weight A^ will only be used for Borel B of Y-shape. Note that = A* and we 
shall denote A* := A® s \ 

Given a Borel 23 of |-shape, we define s(B) to be the sign of TYJLi Vj- 

Lemma 6.8. [K2] The irreducible osp(2m|2n) -module of integral highest weight of the 
form Y27=i fc^i + Y^jLiT^j^j with respect to the standard Borel subalgebra is finite di- 
mensional if and only if(fii, . . . , /i n , . . . ,Jt m ) is either A* or A* for some (n\m)-hook 
partition A. 

We shall denote these irreducible osp(2m|2ra)-modules with respect to the standard 
Borel by L'(osp(2m|2n), A*) and L'(osp(2m|2n), A*). By a similar argument as for 
Theorem 6.4, we establish the following. 

Theorem 6.9. Let A be an {n\m)-hook partition. 

(1) Assume B is of Y-shape. Then, 

(i) A" 2 is the B-extremal weight for the module L / (osp(2?n|2n), A*). 

(ii) A 25 is of the 'B-extremal weight for the module L'(osp(2m|2n), A*). 

(2) Assume B is of \-shape. Then, 

(i) A s is the 'B-extremal weight for L'(o5p(2m\2n), A*) if s(B) = +. 

(ii) X s is the B-extremal weight for L'(osp(2m\2n), A_) if s(B) = —. 
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Corollary 6.10. Form even, every finite dimensional irreducible osp(2?n|2n) -module 
of integral highest weight is self-contragradient. 

Remark 6.11. The remaining S-extremal weights for the modules L / (osp(2m|2n), A*) 
when s('B) = — or for the modules Z/(osp(2m|2n), A^ ) when s(!B) = + are rather 
complicated and do not seem to afford a uniform simple answer. 

The following Dynkin diagram of osp(2m|2n) or g^ that has been in use for super 
duality is opposite to the standard Borel 25 . 




■—c^-^-o o 

a_ 2 a_! /3 1/2 P n -3/2 

Setting ij = e_ m+ j_i and 5i = e n _ i+1 / 2 to match notations, we record the following 
corollary of Lemma 6.8 and Theorem 6.9. 

Corollary 6.12. An irreducible integral h ighes t weight osp(2m|2?i) -module with respect 
to the Borel subalgebra corresponding to ( Qn J is finite dimensional if and only if the 
highest weight is of the form 

m—l n 

(6.2) ±max{A^ - n,0}e_ m - ^ max{A' ? - - n, 0} e_j - K-i+itj-i/2, 

j=i i=i 

where A = (Ai, A 2 , • • •) is an {n\m)-hook partition. 

Remark 6.13. From Corollaries 6.7 and 6.12 it follows that, after passing to the central 
extension g n on which the center K acts as a scalar multiplication by d € Z, the weights 
in (6.1) and (6.2) lie in whenever d < —X\. Hence, Theorem 4.8 and Lemma 3.2 
provide a complete solution to the finite dimensional irreducible character problem for 
the ortho-symplectic Lie superalgebras. 

Remark 6.14. Recall [BR, Sv] that finite dimensional irreducible polynomial g[(n|m)- 
modules are exactly the highest weight modules L'(gl(n\m), A*) with respect to the 
standard Borel subalgebra parametrized by (n|m)-hook partitions A. One can assign 
to any Borel subalgebra S of g[(n|m) an e5 sequence as in Section 6.1, but now with 
£j = rjj = 1, for all i,j. By the same argument as for Theorem 6.4, we can show that 
the highest weights of the polynomial representations of g[(n|m) with respect to 13 is 
given by A s = Y7=iPi S s(i) + TJj=i <7j%)- 

6.3. By flipping from the left to right the Dynkin diagram of QF) and changing all 
the simple roots therein to their opposites, we obtain a Dynkin diagram (°W) corre- 
sponding to the opposite Borel subalgebras, where p = b, b*, c,U. Similarly, by flipping 
the Dynkin diagrams C%n), (T^) and (t^) and changing all signs of the simple roots 
for ngNU {oo}, we obtain the Dynkin diagrams (^Q), (^%) and (^%), respectively, 
of the opposite Borel subalgebras. We form the diagrams corresponding to the Borel 
subalgebras opposite to (2.2) as follows: 

® — ffi — fD — 
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The corresponding Lie superalgebras are again g, g and g, respectively. 

The arguments in Sections 3 and 4 can be adapted easily to allow us to compare 
correspondingly defined parabolic categories °0, °0 and °0 using these opposite Borel 
subalgebras, whose precise definitions are evident. We note that for the corresponding 
set of weights °P + of the form 

m 

J^Aie-i-^Atey + dAo, d G C, 

i=l jeN 

to satisfy the corresponding dominant condition we require, besides the obvious domi- 
nant condition on the standard Levi subalgebra of (°W), also that A + = (AJl , Ajj~, . . .) is 
a partition. This allows us to prove an analogous version of Theorem 4.8 and thus to 
compute irreducible characters of Lie superalgebras in terms of irreducible characters 
of Lie algebras. Also the results in Section 4.5 and section 5 have fairly straightfor- 
ward analogues in °0, °0 and °0 as well. In particular, we can prove equivalences 
of the corresponding finitely generated module subcategories following the strategy of 
Section 5. 

Besides of its own interest, another virtue of this opposite version of super duality lies 
in the ease of calculation of finite dimensional irreducible characters of modules over the 
finite dimensional ortho-symplectic Lie superalgebras. As the highest weight modules 
over g in this setup already have highest weights over the standard Borel subalgebras, 
the knowledge of extremal weights for finite dimensional irreducible modules is no longer 
needed to imply that solution of the irreducible character problem in the category °0 
and °0 n also solves the finite dimensional irreducible character problem. 
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